Multiresolution analysis and matrix factorization are foundational tools incomputer vision. In this work, we study the interface between these twodistinct topics and obtain techniques to uncover hierarchical block structurein symmetric matrices -- an important aspect in the success of many visionproblems. Our new algorithm, the incremental multiresolution matrixfactorization, uncovers such structure one feature at a time, and hence scaleswell to large matrices. We describe how this multiscale analysis goes muchfarther than what a direct global factorization of the data can identify. Weevaluate the efficacy of the resulting factorizations for relative leveragingwithin regression tasks using medical imaging data. We also use thefactorization on representations learned by popular deep networks, providingevidence of their ability to infer semantic relationships even when they arenot explicitly trained to do so. We show that this algorithm can be used as anexploratory tool to improve the network architecture, and within numerous othersettings in vision.
展开▼
